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Can Report Templates Aid Student Learning in Undergraduate Chemistry 
Laboratory Classes? 


Abstract 

This paper describes a study aimed at assessing the ability of report templates to help students learn key 
concepts during undergraduate laboratory classes. The report templates were designed so that a set of 
assessment questions led the students through the logical steps required to perform the laboratory exercise 
and to calculate the required quantitative results. Additional questions probed the students’ understanding of 
the key concepts covered. 

The study assessed the efficacy of the report templates in achieving these goals over a 3 year period via the use 
of a questionnaire, laboratory marks and marks awarded for examination questions that tested one of the key 
learning outcomes. The report templates were found to save time for students and academics alike without any 
negative (or positive) effect on learning outcomes. 
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Introduction 

In a typical undergraduate chemistry course, approximately half the student contact hours take 
place within the teaching laboratory (Royal-Australian-Chemical-Institute 2005). In an ideal 
situation, a laboratory class can be popular with the students (Deters 2005) and an excellent 
vehicle for quality learning where students demonstrate the main concepts for themselves 
(Teixeira-Dias et al. 2005). This is experimental learning in its purest form, which has been shown 
to be a highly effective form of learning (Boud et al. 1994; Taconis et al. 2001). However, several 
studies have shown that chemistry students often fail to connect practical laboratory-based 
exercises with material and concepts covered in lectures and tutorials (Boud et al., 1994; Buntine 
et al. 2007). This may be due to students having a limited “working memory” - a finite capacity to 
process new ideas and work through the practicalities of a laboratory exercise (Cavanagh 1972; St 
Clair! Thompson et al. 2010). Unfortunately, the consequence can be that some students miss the 
key ideas because their working memory is already filled by the mechanics of carrying out the 
experimental procedure of the laboratory exercise (Vianna et al. 1999). Students have an ever- 
increasing range of academic styles and abilities, with research showing that introverted 
personalities have better academic outcomes in chemistry than other personalities; however, 
introverts may struggle with the group work required in many laboratory classes (Schmid et al. 
2009). 

Assessment of laboratory class activities usually involves students marking reports they’ve written 
they have left the laboratory. This may exacerbate the problems of students not relating the 
practical activities to the main concepts because they do not start answering assessed questions 
until they have finished the practical class. Since research has shown that students are primarily 
motivated to learn what is required to pass or do well in assessment tasks (Biggs 2003), it is best 
practice for assessment questions to be focused on the desired learning outcomes, an approach 
recommended in the literature (Biggs 2003; Hall 2002). 

Significant effort has been applied recently in an attempt to achieve improved learning outcomes 
in the undergraduate chemistry laboratory, including improvements in experimental design and 
testing (Buntine et al. 2007) and in the learning environment, including pre-lab exercises, 
identification of learning goals, problem-solving and assessment (Duis et al. 2013; Kelly & 
Finlayson 2007; Wenzel 2003). Previous research has shown that pre-lab work is effective in 
increasing students’ capacity to understand concepts in laboratory classes by reducing the working 
memory required, and also results in improved attitudes about the laboratory course (Pogacnik & 
Cigic 2006; Vianna et al. 1999). Rubrics have also been shown to improve the consistency of 
marking and to help teaching staff identify inadequacies in students’ understanding, thereby 
directing their teaching efforts towards the identified learning goals (Chen et al. 2013;Timmerman 
et al. 2011). 

This study aimed to test the effectiveness of report templates (which incorporate a marking rubric) 
in helping students achieve their primary learning outcomes from laboratory exercises. 


Assessment of Undergraduate Chemistry Laboratory Classes 


This study was undertaken within the laboratory component of a third-year analytical-chemistry 
subject known as “Instrumental Analysis”. The primary learning objective of the practical 
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exercises is for the students to gain a detailed understanding of each of the instruments and the 
physical and chemical properties that the instruments measure. As with other chemistry subjects at 
the University of Wollongong, a practical manual containing all the information required to 
complete the laboratory component of the subject was made available to students at the start of the 
semester. The practical manual outlined the requirements for each laboratory exercise, including 
pre-lab calculations, descriptions of the analysis to be undertaken and the instruments provided, 
and a series of questions to be answered as part of the assessment. Students used duplicate 
notebooks to record all results within the undergraduate laboratory classes so they could hand in 
their raw data whilst retaining a copy for their records. 

Up through 2009, all laboratory exercises had been assessed by “open report”, where students 
were asked to submit a formal scientific report of their findings. Students were instructed to 
include in their reports a brief aim; an introduction outlining the chemical principles on which the 
practical was based; a method summary; a results summary (with original data as an appendix); a 
procedure that detailed the instrument parameters used; a discussion of the results (which 
incorporated their answers to specific questions posed); and finally a conclusion and references. 
This method of assessment was used as one of the main vehicles for students to learn scientific 
report-writing skills, a key learning outcome for students majoring in chemistry. However, many 
students only poorly met this learning objective, possibly because the open-report assessments 
failed to motivate the students to hone their writing skills. This was because the vast majority of 
the marks were awarded for achievement of the specific learning objectives set for each individual 
laboratory exercise. In the two examples used in this study, either no marks or only 5% of marks 
were awarded for the overall quality and presentation of the report. This meant that students had 
little or no incentive to invest time in learning the skills of clear report writing. 

The aim of this study was to determine whether the key learning objectives of the instrumental- 
analysis subject could be more efficiently met by introducing report templates that encouraged 
students to answer set questions during laboratory classes. It was hypothesised that this would help 
the students relate the practical activities to the key concepts being taught and would save time 
(for both students and teachers) that could be used in future for an additional assessment focused 
specifically on scientific-writing skills. 


Designing Report Templates 

In 2010, the assessment format was altered from an open report to a template for two out of five 
the existing laboratory exercises within the third-year undergraduate instrumental-analysis subject. 
The remaining three laboratory exercises were assessed by open report as in previous years so that 
the students’ reactions to the change could be gauged by comparison. 

The new assessment templates resembled worksheets, with a set of questions that led the students 
through logical steps to highlight the main concepts and learning objectives of each laboratory 
exercise. They were designed to be answered in part as the students work through the laboratory 
exercises in class. The templates were provided as both hard copy and soft copy, and the students 
were told how many marks each question was worth and an indication of the length of answer 
expected (from the size of the space allocated for their answer in the hard-copy template). 

The two exercises templated were “Gas Analysis by Fourier Transform Infra-Red Spectroscopy” 
(FTIR) and “Measuring Polycyclic Aromatic Hydrocarbons in Sediments using Gas 
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Chromatography Mass Spectrometry” (GC-MS). The laboratory exercises and templates were 
provided as supplementary material to the students. 

The marking schemes for the previous open-report assessment of these laboratory exercises had 
awarded marks for correctly working through the required calculations. Marks were also awarded 
for answering a series of relevant questions aimed at probing students’ understanding of the main 
concepts and learning objectives. The marking schemes used for the templated reports awarded the 
same marks for the calculations and questions, again with either zero or 5% of marks awarded for 
quality of presentation. 

It was envisaged that the report templates could be completed within the laboratory class and 
submitted as the students left; however, only a handful of students did this. The vast majority of 
students submitted the templates the following week within the same time limit as that set for the 
open reports. Students were encouraged to use their duplicate laboratory notebooks as usual and to 
hand in raw results along with their completed templates. 


Testing the Effectiveness of the Report Templates 

Over a three-year period (2010-2012 inclusive), evidence was gathered to assess the success of the 
templates in improving the achievement of student learning outcomes. Three means were used to 
assess the effectiveness of the templates. The first was a student questionnaire to gather feedback 
on the students’ perceptions of how the templates had altered their learning experience. The 
second was a simple comparison of the marks achieved by students using the templates compared 
to the work of students from previous years that had been assessed via open-format reports. The 
final method was an examination question (Box 1) that tested how well the students had retained 
knowledge of two different analytical techniques, both of which were the subject of laboratory- 
class exercises, but only one of which (GC-MS) had been assessed by a report template. This 
tested the first of the key learning objectives for the practical exercises, which was for students to 
understand the analytical instruments’ basic principles of operation. The same examination 
question appeared in the final examination in 2010 and at the end-of-semester practical laboratory 
examinations in 2011 and 2012. 


Results and Discussion 

The questionnaire 

Students were given six statements regarding the reports and asked to rank the statements on the 
following scale: +3 (strongly agree), +2 (agree), +1 (mildly agree), -1 (mildly disagree), -2 
(disagree), and -3 (strongly disagree). Table 1 shows the questions and the mean response of the 
respondents in each year from 2010 to 2012, and the total number of students who completed the 
survey each year; this number expressed as a percentage of all students who were enrolled in the 
subject on the survey date. The final column of Table 1 gives the mean of all respondents across 
all years (a total of 64 respondents out of a possible 77 students enrolled in the subject between 
2010 and 2012). 
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Overall the survey showed the templates to be reasonably popular with students, with average 
results for all questions falling between the “agree” and “mildly agree” categories. Most students 
found the templates easy to complete, and reported attempting some of the questions during the 
allocated laboratory time. In addition, students reported spending less time overall on the 
templated reports than on the open reports. Despite this, most students believed that the templates 
helped them to better understand and remember the key points from the laboratory exercises. The 
majority of students were reasonably happy with the idea of using report templates for all 
laboratory assessments. 


Box 1: Examination question testing knowledge of instrumentation studied in a third-year 
analytical-chemistry course and the marking scheme used to grade answers _ 


Describe what happens to a sample once it is injected into each of the listed instruments. Also 
include a description of the physical properties that result in separation, the detection method(s) 
employed and how the instrument was used to yield qualitative and quantitative measurements of 

the analyses. 


U) 

Ion Chromatography (IC) 


(ii) 

Gas Chromatography Mass Spectrometry (GCMS) 


Chromatography 


I. 

In your practical classes you used different instruments that involve chromatographic 


separation techniques, including: 



(a) Ion Chromatography (IC) - from the Analysis of Anions practical 


a. 

Injection valve - ensures same amount injected 

(0.5) 

b. 

Guard column 

(0.5) 

c. 

Main column - separation by ion-exchange (positively charged groups 

on the stationary 


phase attract solute ions and slow their elution time depending upon their charge and size) 



(1) 

d. 

Suppressor - chemically reacts with the carrier eluent so only analyses conductivity 


remains 

(1) 

e. 

Conductivity detector 

(0.5) 

f. 

Qualitative by comparison to retention times 

(0.5) 

g- 

Quantitative by calibration with known standards 

(0.5) 

h. 

Bonus mark for quality of description/extra information 

(0.5) 


(b) Gas Chromatography Mass Spectrometry (GCMS) - from the PAH 
sediments practical 

in 

a. 

Split or split-less injection 

(0.5) 

b. 

Column - separation by boiling points 

(0.5) 

c. 

Ionisation (electron-spray) 

(0.5) 

d. 

Separation by mass-to-charge ratio 

(0.5) 

e. 

Detection by current 

(0.5) 

f. 

Qualitative - by retention times & reference library of fracture patterns 

(1) 

g- 

Quantitative by use of internal standard 

(1) 

h. 

Bonus mark for quality of description/extra information 

(0.5) 
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The marks for laboratory reports 

Table 2 shows the mean and standard deviations of marks awarded for the completed laboratory 
reports for the GCMS and FTIR laboratory exercises in 2009 (when these exercises were assessed 
by open report) compared to 2010-2012 (when these exercises were assessed by report template). 
The final row of Table 2 gives the mean results for all three years that used the report templates, 
whilst the last column of Table 2 provides the mean and standard deviation of the final overall 
subject marks that students obtained in each year of the study. The final mark is composed of 20% 
for the practical reports; 20% for the practical exam; 10% for a scientific poster presentation; and 
50% for the final exam. The final subject marks represent our best indicator of the overall ability 
of the cohort, and from this statistic the 2009 cohort appears marginally stronger than the 2010- 
2012 cohorts. 

Overall the results show a slight increase in the marks awarded to the students who used the report 
templates compared to those who completed the open reports. The difference is not large but was 
achieved despite the slightly poorer overall course results obtained by the 2010-2012 cohort 
compared to the 2009 cohort. 

The slight increase in marks for the GCMS exercise indicates an improvement in the number of 
students working through the extraction-efficiency calculation correctly, probably because the 
layout of the template led them through the necessary steps. There was no single aspect of the 
FTIR practical that was better answered by students using the report templates than those using the 
open report, and the difference in total marks achieved was small. 

Due to the large standard deviation in the marks awarded each year for the laboratory exercises 
and in the final course marks, it is difficult to draw immediate conclusions about the statistical 
significance of these results. For this reason, an F-test was performed on the data to help identify 
any differences that were statistically significant. 

A two-sided F-test was performed whereby the ratio of the variance of each sample was used to 
both samples were from the same population (i.e., that there was no 

Si- 


test the hypothesis that 
statistical difference). 

The F-statistic is: F — 


where is the variance of sample x and is the variance of sample y. The larger variance is 
used as the numerator and the smaller variance as the denominator, such that the F-statistic is 
always greater than one. For this reason the distribution is two-sided and the F-statistic limit at 
0.025 is equivalent to the 5% significance level. The F-statistic limit depends on the number of 
values in each sample, and may be looked up in a table of values (Chatfield 1983). 


If the F-statistic is larger than the F-statistic limit, there is only a 5% chance that these samples are 
from the same population, so it can be concluded that the difference is likely to be significant. If 
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on the other hand the F-statistic is below this value, the differences are deemed not to be 
significant. 


http://ro.uow.edu.au/jutlp/voli2/iss2/3 


6 



Paton-Walsh: Can Laboratory Report Templates Aid Student Learning? 


Table 1: Results of the questionnaire regarding students’ perceptions of the laboratory-report templates ranked on the 
following scale: +3 (strongly agree), +2 (agree), +1 (mildly agree), -1 (mildly disagree), -2 (disagree), and -3 (strongly disagree). 


Year 

2010 

2011 

2012 

Mean 

Number of Respondents (% of students enrolled on survey date) 

16 (70%) 

19 (95%) 

29 

(83%) 

64 

(83%) 

The laboratory templates were easy to complete 

1.9 

1.9 

1.5 

1.7 

When I used the templates I completed some of the questions during the laboratory time 

1.3 

1.8 

1.8 

1.7 

Using the templates helped me to understand the main points better 

1.8 

1.8 

1.7 

1.8 

I spent less time in total completing the templated reports than on the other reports 

2.2 

1.9 

1.7 

1.9 

It is easier to remember key points from the laboratory reports that were templated 

1.8 

1.1 

1.1 

1.3 

I would prefer to use templates to report on all my laboratory work 

1.8 

1.7 

1.0 

1.4 


Table 2: Mean and standard deviations of marks awarded for the completed laboratory reports for the GCMS and FTIR 
la boratory exercises in 2009 (open report) compared to 2010-2012 (report template). __ 



GCMS 

FTIR 

Final Course Mark 

2009 marks (open report - 29 students) 

(70 + 19)% 

(75 + 17)% 

(73 + 12)% 

2010 marks (using report template - 28 students) 
difference to 2009: Significant? Yes/No 

F-test (5% significance level for F-statistic = 2.16) 

(74 + 19)% 
No 

1.04 

(75 + 13)% 
No 

1.79 

(67 + 11)% 

No 

1.12 

2011 marks (using report template - 20 students) 
difference to 2009: Significant? Yes/No 

F-test (5% significance level for F-statistic = 2.39) 

(86 + 8)% 
Yes 

5.16 

(84 ± 12)% 
No 

2.19 

(69 + 16)% 

No 

1.73 

2012 marks (using report template - 30 students) 
difference to 2009: Significant? Yes/No 

F-test (5% significance level for F-statistic = 2.11) 

(79 + 17)% 
No 

1.73 

(80+10)% 

Yes 

2.42 

(66 + 16)% 

No 

1.76 

Average marks awarded for templated reports 2010-2012 (78 students) 
2010-2012 difference to 2009: Significant? Yes/No 

F-test (5% significance level for F-statistic = 1.78) 

(79 + 16)% 
No 

1.31 

(80+12)% 

Yes 

1.95 

(67 + 14)% 

No 

1.43 
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The results of the two-sided F-tests are also given in Table 2 for each year in 2010-2012 compared 
to the control year 2009, along with the 5% significance limit for the F-statistic for the numbers in 
each sample. For individual years when the templates were used, only the GCMS results from 
2011 and the FTIR results from 2012 were significantly better than the 2009 control (no 
templates). When the 2010-2012 results were combined, the differences in the GCMS results and 
the final course marks were deemed insignificant, whilst the slight improvement in FTIR results 
was deemed significant. 

Thus the apparently larger improvement in marks in the GCMS reports was actually not 
statistically significant, and the only measurable effect of the templates was a small improvement 
in the FTIR report marks. Nevertheless, the main learning objectives (to understand the basic 
principles of operation of the analytical instruments), were met at least as well with the report 
templates as with the open reports, and with less time apparently required from the students. 
Marking was also simplified, so that academics also saved time with the use of templates. 


The final-exam question 

A more robust test of whether the learning objectives were achieved equally well using and open 
reports is how well the information and understanding were retained by students. This was tested 
by comparing marks awarded for parts (a) and (b) of the exam question shown in Box 1. The 
students’ knowledge and understanding of ion chromatography (assessed by open report) was 
compared to that of gas chromatography-mass spectrometry (assessed by report template). The 
results for 2010-2012 are given in Table 3. 


Table 3: Mean and standard deviations of marks awarded for the final-exam question 
comparing students’ retention of information learnt during the laboratory exercises on ion 
chromatography (open report) and GC-MS (report template) 



Final Exam 2010 

Practical Exam 2011 

Practical Exam 2012 

IC Question (open report) 

(57 + 20)% 

(49 + 21)% 

(49 ± 23)% 

GCMS Question (template) 

(60 + 30)% 

(52 + 23)% 

(49 + 25)% 


Both parts of the question were answered quite poorly overall, with the average mark for both 
being around the pass rate of 50%. Whilst it is common for descriptive questions to score more 
poorly than calculation questions in this subject, these results are disappointing. The only obvious 
difference in the results shown in Table 3 is that the questions were answered better in 2010, when 
they appeared in the final exam, than in 2011 and 2012, when they appeared in the end-of- 
semester practical exam. This probably has more to do with students’ revision habits than any 
impact of the assessment style during the course. 

Throughout the three years, there was no statistical difference between the quality of answer 
provided by the students for the ion chromatography section (assessed by open report) and that 
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provided for the gas chromatography-mass spectrometry section (assessed by report template). 
This was demonstrated by applying a paired t-test (Chatfield 1983) of the examination results. 

This is a statistical test of the hypothesis that there is no difference between two sets of data. If this 
hypothesis is true, then the set of differences of students’ marks for the questions will be a sample, 
size n, from a normally distributed population with mean zero. Thus, for each student, the marks 
awarded for their answer to the ion chromatography (IC) question were subtracted from the marks 
awarded for their answer to the gas chromatography-mass spectrometiy (GCMS) question, to give 
a set of difference values. 


. _ -- 

The t-test statistic is E — -;— 

i £ r/v.v 

where d is the mean of the set of differences, S d is the standard deviation of the set of differences 
and N is the total number of differences (i.e., the number of students sitting the final 
examinations). At the 95% confidence level, t = 1.96 (for infinite n). If t is less than this it can 
be concluded that the hypothesis that there is no systematic difference between the two sets of 
marks is most probably correct. For all three years combined, the mean difference and its 
standard deviation was 0.08 + 1.3 for a total of 72 students, yielding a t-statistic of 0.5. Thus it can 
be concluded that the templates neither increased nor decreased the students’ quality of answers in 
the final examination. This implies that the use of templates had no statistically measurable impact 
on the students’ ability to retain the information learned during the laboratory exercises. 


Student Feedback 

The results of the survey showed that the majority of students believed that the templates helped 
them to remember the key points from the laboratory exercises. Despite this, the examination 
results showed that there was no measurable improvement in performance as a result of using the 
templates. This implies that any perceived benefit either was not real or did not contribute to deep 
learning and so the benefit was not retained. 

An attempt was made to contact the students via their old university email addresses. The students 
were told that the templates had failed to improve the examination performance, and were asked: 
“Given this information, what is your opinion now about the templates?” In addition, they 
were asked to report their level of agreement (in hindsight) with two of the original survey 
questions: 

1. Using the templates helped me to understand the main points better 

2. It is easier to remember key points from the laboratory reports that were templated 

Only six replies were received (and only five answered the survey questions), but amongst these 
were several interesting reflections (Box 2). There remained a small positive attitude towards the 
templates in the few responding students (a mean of 0.75 for the first question and 1.0 for the 
second), but no conclusions should be drawn from this because the sample size is so small and 
there may be a bias in those who chose to reply. 
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Box 2: A number of interesting reflections provided by students who used the templates 
(note that these comments were provided 18 months or longer after original use of templates) 


1 

“Well 1 think that most students treat a lab report and template like a hurdle they need 
to jump through rather than a learning experience. They will not think about the long 
term learning outcomes. They'll deal with relearning before the exam. So while a 
template at the time may help to the student to focus on the main points of the 
exercise, all this effort is lost as soon as the report is handed in. Any advantages or 
disadvantages due to the effect of the template is long lost when assessment is 
done.” 

2 

“.using a template tells the student the items that are important and those that he 

needs to know. Its shortcoming is that the student may be tempted to just fill it out 
without giving it much thought (e.g. connecting different concepts together, 
synthesizing the key points). 1 guess one way to address the gap is to conduct follow¬ 
up discussions after the practical session.... This would also provide a venue for the 
teachers to emphasize the key points even further.” 

3 

“The only issue which 1 had with actually using the templates, was the fact that for 
some reason 1 didn't study them for the practical exam! So, all in all, it was my plain 
stupidity and ignorance that let me down. Not the template! 1 did actually find the 
template quite helpful at the time of the experiment.” 

4 

“1 did not find the templated lab questions to aid my understanding of the material. To 
be quite honest, 1 remember feeling happy about submitting templated questions, 
because 1 perceived them as involving less work than a full prac report. But for me, 
tasks involving less work usually meant less understanding.” 

5 

“1 think it was easier to revise before the exam, more than easier to remember.” 

6 

“Without looking at the student learning there are still benefits with regards to 
marking and paperwork. It also benefits the student in knowing the length of answer 
needed given the space allocated for each answer and the marks allocated to each 
answer (from memory they were there also).” 


There are some common themes in the students’ reflections (Box 2) - especially that the templates 
were seen to impart a short-term benefit, as key points were identified and the reports could be 
completed with less work than required for the open reports. However, students also equated less 
work with less learning. Without further revision or reinforcing, deep learning was not achieved 
and there were no long-term benefits in the use of templates over open reports. 

A very interesting point was raised by one student about the lack of pre-lab questions in the 
templates. The students were asked to read through the laboratory exercises in preparation for the 
class, but because no assessable pre-lab questions were set, it is likely that many did not bother. 
The importance of pre-lab material is well recognised in the literature (Pogacnik & Cigic 2006; 
Vianna et al. 1999), as well as by this astute student! It is recommended that in future, the 
templates be redesigned to include a pre-lab exercise that introduces the students to the main 
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components and operation of the instruments before they arrive in the undergraduate teaching 
laboratory. 

Finally, one student raised the point about time saved in marking and paperwork. This is also 
important because if the same learning outcomes can be achieved in a shorter time frame, it may 
be possible to find time for a formal assessment of scientific writing. 


Summary and Conclusions 

Students are primarily motivated to learn what is required to pass or do well in assessment tasks 
(Biggs 2003), so marks for laboratory exercises should be focused on the desired learning 
outcomes. Report templates aim to help achieve this goal and improve students’ ability to connect 
practical laboratory-based exercises with the key concepts by allowing them to answer questions 
during laboratory classes. Clearly there is also a requirement for students to learn both the skills 
inherent in producing a detailed report of laboratory methods and results and how to correctly 
interpret their findings. However, these learning objectives are often poorly met by existing open- 
report assessments. 

This study attempted to assess the success of report templates in aiding student learning in an 
undergraduate analytical-chemistry laboratory classes. Over three years, a questionnaire showed 
that the report templates were popular with the students, who reported spending less time 
completing them than they spent on open reports. The students believed at the time that the 
templates had helped them to better learn and remember key points. Whilst this did not translate 
into any actual improvement in exam results, the students performed just as well in answering a 
question about the laboratory exercise that had been assessed by a report template as one that had 
been assessed by an open report, despite the reduced time spent. 

Report templates also provided a time-saving benefit to teaching staff because it was easier to 
apply a consistent marking scheme, thereby reducing the marking time required. Thus it may be 
concluded that time for a supplementary assessment focused specifically on writing skills could be 
found if the majority of student laboratory exercises were assessed via report templates. Since the 
use of templates did not adversely affect students’ results, this could be achieved without 
compromising other key learning outcomes specific to the subject. 
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